Video monitoring apparatus, control method thereof, and computer readable medium

ABSTRACT

A position and pose determination unit ( 103 ) is configured to determine a position and an pose of the video acquisition unit ( 110 ) in such a way that a level of hiding caused by overlapping of objects becomes low based on environmental object information indicating an environmental object including a structural object present in the monitored area and a placed object placed in the structural object, and staying characteristic information indicating a staying characteristic of the object determined depending on the environmental object. The analysis and display unit ( 111 ) is configured to perform at least one of analyzing a video captured by the video acquisition unit ( 103 ) at the position and the pose determined by the position and pose determination unit ( 110 ) and displaying the video.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application is a continuation application of U.S. patentapplication Ser. No. 17/741,607 filed on May 11, 2022, which is acontinuation application of U.S. patent application Ser. No. 16/981,898filed on Sep. 17, 2020, which issued as U.S. Pat. No. 11,381,782, whichis a National Stage Entry of international application PCT/JP2018/013230filed on Mar. 29, 2018, the disclosures of all of which are incorporatedin their entirety by reference herein.

TECHNICAL FIELD

The present disclosure relates to a video monitoring apparatus thatmonitors a video using a video acquisition unit.

BACKGROUND ART

Various video monitoring apparatuses for monitoring videos captured by acamera have been proposed. For example, Patent Literature 1 discloses atechnique of arranging a plurality of cameras in a monitored area, anddetecting and tracking an object with the cameras. The techniquedisclosed in Patent Literature 1 presents a behavior state of the objecton a monitoring area of a monitoring screen based on a result of thetracking.

On the other hand, Patent Literature 2 discloses a technique ofevaluating whether a camera layout is suitable for video processing whencameras are placed in a monitored area. According to the techniquedisclosed in Patent Literature 2, it is evaluated whether the cameralayout (an orientation and position of the camera) is suitable forspecific video processing based on whether an object is hidden behind anobstacle, a resolution, and an angle of a gaze point when viewed fromthe camera and then displays a result of the evaluation. It is possibleto construct a monitoring apparatus suitable for video processing suchas human detection by designing the monitoring apparatus using thetechnique disclosed in Patent Literature 2.

CITATION LIST Patent Literature

-   Patent Literature 1: Japanese Patent No. 3257165-   Patent Literature 2: International Patent Publication No. WO    2014/002398

SUMMARY OF INVENTION Technical Problem

However, Patent Literature 1 does not disclose a camera layout suitablefor monitoring an object appearing in a video.

Further, the technique disclosed in Patent Literature 2 considers astructure of the monitored area and a fixed structural object placedinside the monitored area. However, the technique disclosed in PatentLiterature 2 does not consider about information such as where an objectto be monitored is likely to stay and where and how much objects arehidden, and thus does not consider about the possibility in which theobjects overlap with each other and the camera layout becomes unsuitablefor video processing. For this reason, with the technique disclosed inPatent Literature 2, the camera layout may not be necessarily suitablefor video monitoring under a situation where a plurality of objects arepresent inside the monitored area and overlap with each other.

An example object of the present disclosure is to solve one of theabove-described problems and to provide a video monitoring apparatusthat can monitor a video more appropriately even under a situation whereobjects overlap with each other in a monitoring area.

Solution to Problem

In an example aspect, a video monitoring apparatus for monitoring insidea monitored area using a video acquisition unit. The video monitoringapparatus includes:

-   -   a position and pose determination unit configured to determine a        position and an pose of the video acquisition unit in such a way        that a level of hiding caused by overlapping of objects becomes        low based on environmental object information indicating an        environmental object including a structural object present in        the monitored area and an placed object placed in/on the        structural object and staying characteristic information        indicating a staying characteristic of the object determined        depending on the environmental object; and    -   an analysis and display unit configured to perform at least one        of analyzing a video captured by the video acquisition unit at        the position and the pose determined by the position and pose        determination unit and displaying the video.

In another example aspect, a method of controlling video monitoringapparatus for monitoring inside a monitored area using a videoacquisition unit. The method includes:

-   -   determining a position and an pose of the video acquisition unit        in such a way that a level of hiding caused by overlapping of        objects becomes low based on environmental object information        indicating an environmental object including a structural object        present in the monitored area and an placed object placed in/on        the structural object and staying characteristic information        indicating a staying characteristic of the object determined        depending on the environmental object; and    -   performing at least one of analyzing a video captured by the        video acquisition unit at the determined position and the        determined pose and displaying the video.

In another example aspect, a non-transitory computer readable mediumstores a program for causing a computer to monitor inside a monitoredarea using a video acquisition unit. The program causes the computer toexecute:

-   -   a procedure of determining a position and an pose of the video        acquisition unit in such a way that a level of hiding caused by        overlapping of objects becomes low based on environmental object        information indicating an environmental object including a        structural object present in the monitored area and an placed        object placed in/on the structural object and staying        characteristic information indicating a staying characteristic        of the object to be monitored determined depending on the        environmental object; and    -   a procedure of performing at least one of analyzing a video        captured by the video acquisition unit at the determined        position and the determined pose and displaying the video.

Advantageous Effects of Invention

According to the above example aspects, it is possible to provide avideo monitoring apparatus that can monitor a video more appropriatelyeven under a situation where objects overlap with each other in amonitoring area.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram showing a configuration example of a videomonitoring apparatus 100 according to a first embodiment of the presentdisclosure;

FIG. 2A is a view showing a camera layout example of cameras forcapturing persons lining up at a cash register counter;

FIG. 2B is a view showing an example of a video assumed when the videois captured with a camera layout (A) of FIG. 2A;

FIG. 2C is a view showing an example of a video assumed when the videois captured with a camera layout (B) of FIG. 2A;

FIG. 3A is a top view of the camera layout (A) of FIG. 2A;

FIG. 3B is a top view of the camera layout (B) of FIG. 2A;

FIG. 4 is a side view of the camera layouts (A) and (B) of FIG. 2A;

FIG. 5A is a view showing a camera layout example of cameras forcapturing persons lining up at a cash register counter where a pluralityof queues are formed;

FIG. 5B is a view showing an example of a video assumed when the videois captured with a camera layout (A′) of FIG. 5A;

FIG. 5C is a view showing an example of a video assumed when the videois captured with a camera layout (B′) of FIG. 5A;

FIG. 6A is a view showing a camera layout example of cameras forcapturing persons lining up at a cash register counter where a pluralityof queues are formed;

FIG. 6B is a view showing an example of a video assumed when the videois captured with a camera layout (A″) of FIG. 6A;

FIG. 6C is a view showing an example of a video assumed when the videois captured with a camera layout (B″) of FIG. 6A;

FIG. 7 is a graph showing an example of a relation between a hidingduration and a temporal hiding level;

FIG. 8A is a view showing a camera layout example of cameras forcapturing persons in a passage in which one side thereof is a wall and ashelf is placed on the other side thereof;

FIG. 8B is a view showing an example of a video assumed when the videois captured with a camera layout (C) of FIG. 8A;

FIG. 8C is a view showing an example of a video assumed when the videois captured with a camera layout (D) of FIG. 8A;

FIG. 9A is a view showing a camera layout example of cameras forcapturing persons in a passage in which shelves are placed on both sidesthereof;

FIG. 9B is a view showing an example of a video assumed when the videois captured with a camera layout (C′) of FIG. 9A;

FIG. 9C is a view showing an example of a video assumed when the videois captured with a camera layout (D′) of FIG. 9A;

FIG. 10 is a block diagram showing a configuration example of a videomonitoring apparatus 200 according to a second embodiment of the presentdisclosure; and

FIG. 11 is a block diagram showing a configuration example of a videomonitoring apparatus 300 according to a third embodiment of the presentdisclosure.

DESCRIPTION OF EMBODIMENTS

Hereinafter, embodiments of the present disclosure will be describedwith reference to the drawings. The following descriptions and drawingsare omitted and simplified as appropriate for clear descriptions.Further, in the following drawings, the same elements are denoted by thesame signs, and repeated descriptions are omitted as necessary.

Referring to FIG. 1 , FIG. 1 shows a configuration example of a videomonitoring apparatus 100 according to a first embodiment of the presentdisclosure. The video monitoring apparatus 100 includes a videoacquisition unit 110, an analysis and display unit 111, an environmentalobject information storage unit 101, an environmental object dependentobject staying characteristic information storage unit 102, and aposition and pose determination unit 103.

The video acquisition unit 110 captures a video of a monitored area andoutputs the captured video to the analysis and display unit 111.

The analysis and display unit 111 performs at least one of analyzing thevideo output from the video acquisition unit 110 and displaying it.

Here, the video acquisition unit 110 is installed in a monitoring areaor in the vicinity thereof to capture the inside of the monitoring area.A position and pose of the video acquisition unit 110 are determinedbased on video acquisition unit layout information determined by theposition and pose determination unit 103.

The position and pose determination unit 103 determines the above videoacquisition unit layout information based on the environmental objectinformation of the monitoring area stored in the environmental objectinformation storage unit 101 and environmental object dependent objectstaying characteristic information stored in the environmental objectdependent object staying characteristic information storage unit 102.

Next, an operation of the video monitoring apparatus 100 shown in FIG. 1will be described.

The video acquisition unit 110 is a device for capturing the inside ofthe monitoring area. Various cameras used for video monitoring such asanalog cameras and IP (Internet Protocol) cameras (network cameras) canbe used as the video acquisition unit 110. In addition to a visiblelight camera, a camera of a wavelength other than visible light, such asan infrared camera can also be used according to the application as thecamera. In the following descriptions, the video acquisition unit isalso simply referred to as a camera.

Here, the layout (position and pose) of the video acquisition unit 110is determined based on the video acquisition unit layout informationdetermined by the position and pose determination unit 103, which willbe described later. The video acquisition unit layout information isrelated to the position and pose of the video acquisition unit 110placed in a certain monitored area. The video acquisition unit layoutinformation includes information such as the position and aninstallation height of the video acquisition unit 110, an orientation ofthe video acquisition unit 110 (the pose of the video acquisition unit110), and a zoom amount (how wide the angle of view is) in the monitoredarea. The video acquisition unit layout information is described inassociation with environmental object information, which will bedescribed later. For example, when the environmental object informationis supplied as drawing information of the monitored area, the videoacquisition unit layout information may be represented by coordinates ofthe drawing. Alternatively, the video acquisition unit layoutinformation may be represented in a real world coordinate system definedwith a certain point on a real space as an origin. Information such asthe orientation and a zoom ratio of the video acquisition unit 110 ofthe video acquisition unit layout information may be described by acamera parameter which determines this information. Note that theinstallation and adjustment of the video acquisition unit 110 based onthe video acquisition unit layout information may be performedautomatically or manually. The video captured by the video acquisitionunit 110 is output to the analysis and display unit 111.

The analysis and display unit 111 presents the video input from thevideo acquisition unit 110 to an observer. Then, the observer can know asituation of a monitored object in the monitored area.

Alternatively, the analysis and display unit 111 may analyze the videoinput from the video acquisition unit 110, perform detection andtracking processing of the object to be monitored, and present a resultof the detection and tracking processing together with the video. Forexample, the analysis and display unit 111 may surround the detectedobject with a rectangle and present it or allocate an ID (identifier) tothe object according to the result of tracking the object, and present avalue of the ID together with the object. Alternatively, the analysisand display unit 111 may detect an abnormal behavior of the object basedon the result of detecting and tracking the object and issue an alert.Further, the analysis and display unit 111 may issue an alert when theobject intrudes into a specific area. These processes may be carried outby various existing methods.

Note that the analysis and display unit 111 may display only the resultof analyzing the video without displaying the video itself. For example,the analysis and display unit 111 may display the position and amovement trajectory of the object on a map based on the result oftracking. Then, when the observer wants to check the video captured bythe specific video acquisition unit 110, the analysis and display unit111 may issue a display processing command to thereby present the videoon the screen.

Next, an operation of the position and pose determination unit 103 willbe described.

The position and pose determination unit 103 receives the environmentalobject information of the monitoring area stored in the environmentalobject information storage unit 101, and the environmental objectdependent object staying characteristic information stored in theenvironmental object dependent object staying characteristic informationstorage unit 102.

Here, the environmental object information is information about astructural object described in a drawing of the inside of the monitoredarea and information about a placed object placed in the structuralobject. Specifically, the structural object is a pillar, a wall, afixture, lighting hanging from the ceiling, an advertisement, a display,a direction board, etc. placed in the monitored area. The structuralobject information is information such as a three-dimensional positionand a size (both the three-dimensional position and the size includeheight information), a shape, an orientation, etc. of the structuralobject placed in the monitored area. The term “placed object” refers toan object placed in/on the structural object placed in the monitoredarea. When the inside of a store is the monitored area, the placedobject is, for example, an item placed in/on a fixture placed as astructural object in the store. The placed object information is relatedto a type of the placed object placed in the structural object. Forexample, when the inside of a store is the monitored area, drawinginformation of a structural object such as a fixture and the like placedin the store and information of a type of an item placed in the fixtureare input as the environmental object information. In the structuralobject information, the information about the position and the height ofthe structural object may be represented by coordinates in the drawingor may be represented in a real world coordinate system.

On the other hand, the environmental object dependent object stayingcharacteristic information is related to a staying characteristic of theobject to be monitored determined depending on the environmental object.For example, when the monitored area is a store, and the object is aperson, persons lining up in a queue are likely to stay in front of acash register inside the store. Moreover, persons browsing tend to stayin front of a book corner inside the store. It can be said that anoccurrence of such a situation depends on structural objects such ascash registers and placed objects such as books. As a result, suchplaces have a staying characteristic in which objects tend to overlapwith each other due to the staying of the objects, and thus objects arelikely to be hidden. The environmental object dependent object stayingcharacteristic information indicates such a staying characteristic. Theenvironmental object dependent object staying characteristic informationmay include, for example, a spatial parameter describing the stayingcharacteristic (information about a staying position, a stayinginterval, or a position and an orientation of a line along which aplurality of staying objects line up (hereinafter referred to as astaying line)), a frequency of staying, and a staying duration.Alternatively, information that defines a value calculated based on thevalue such as the staying duration and represents a staying level may beused as the environmental object dependent object staying characteristicinformation. Further alternatively, hiding level information calculatedbased on the staying characteristic and an index such as a hiding levelmay be used as the environmental object dependent object stayingcharacteristic information. Details of the environmental objectdependent object staying characteristic information will be described inspecific examples.

The position and pose determination unit 103 determines the layout(position and pose) of the video acquisition unit 110 from theenvironmental object information and the environmental object dependentobject staying characteristic information. In this case, the positionand pose determination unit 103 considers the hiding level caused by thestaying of the objects.

The hiding level here represents a level of hiding caused by theoverlapping of the objects and can be divided into a spatial hidinglevel and a temporal hiding level (or a hiding level considering both ofthe levels). The spatial hiding level is an index indicating how much anobject is hidden spatially and can be represented by a ratio of an areaof a hidden part of an object to an entire area of the object appearingin the video (the ratio may be also referred to as a spatial hidingratio). On the other hand, the temporal hiding level is an indexindicating how hidden an object is temporally and can be defined by aduration of the hiding. A total hiding level can be indicated byintegrating the spatial hiding level with the temporal hiding level. Avalue calculated by multiplying the spatial hiding level by the temporalhiding level may be used as the total hiding level. However, theintegration method is not limited to this, and instead any existingintegration method may be used. Specific examples of the spatial hidinglevel and temporal hiding level will be described later.

The position and pose determination unit 103 determines the layout ofthe video acquisition unit 110 so as to lower the hiding level.Hereinafter, the term “layout” includes not only the position of thevideo acquisition unit but also its pose information. For example, theposition and pose determination unit 103 generates a plurality oflayouts of the video acquisition unit 110, and determines a layout inwhich the hiding level becomes the smallest among the plurality oflayouts. Alternatively, the position and pose determination unit 103sets a value to be satisfied by the hiding level as a threshold, andobtains a layout in which the hiding level becomes equal to or less thanthe threshold. Then, the position and pose determination unit 103 maydetermine the most desirable layout according to the other indices amongthe plurality of acquired layouts. Other indices include, for example, aresolution (size) of the object to be monitored in the video, theorientation of the object, and the angle at which the object is captured(a depression angle). In particular, when the video processing isperformed by the analysis and display unit 111, the layout is preferablysuitable for the video processing. Thus, the position and posedetermination unit 103 selects an optimum layout according to thecharacteristic of the video processing. In the following descriptions,the layout of the video acquisition unit is also referred to simply as acamera layout.

The video monitoring apparatus 100 shown in FIG. 1 considers the stayingcharacteristic of the object, obtains the layout of the videoacquisition unit 110 in which the hiding level becomes low, and performsvideo monitoring using the video captured with the acquired layout.Thus, even when a plurality of objects stay, video monitoring can beappropriately performed.

Hereinafter, this embodiment will be described by showing specificexamples.

(Specific Example 1) When the Structural Object of the EnvironmentalObject is a Cash Register Counter

FIG. 2A shows a camera layout example of the video acquisition unit 110(hereinafter also simply referred to as a camera 110) for capturingpersons lining up at a cash register counter, and is a view seen fromdirectly above. Two types of camera layouts (A) and (B) are shown inFIG. 2A. The camera layout (A) shows a camera layout for capturingpersons lining up in front of the cash register counter from the side ofthe cash register counter, and the camera layout (B) shows a cameralayout for capturing the persons lining up at the cash register counterobliquely from the front. FIG. 2B is an example of a video of personsassumed to be captured with the camera layout (A), and FIG. 2C is anexample of a video of persons assumed to be captured with the cameralayout (B). When the assumed video processing is person detection, theless overlap between persons, the easier it is to detect a person.Therefore, when the assumed video processing is person detection, theposition and pose determination unit 103 selects the camera layout (B)between the camera layout examples of FIG. 2A. Hereinafter, a method ofcalculating the spatial hiding level in each case of the camera layout(A) and (B) will be described.

First, the position and pose determination unit 103 generates two cameralayouts (A) and (B) as candidates for the camera layout (position andpose of the camera 110) of the camera 110. The position and posedetermination unit 103 determines a positional relation between thestructural object (in this case, the cash register counter) in themonitored area and the camera 110 from the camera layouts (A) and (B)and the environmental object information as shown in FIG. 2A. Along withthis, the position and pose determination unit 103 can also determinethe position at which customers line up at the time of checkout, and apositional relation between this position and the camera 110 (thedistance from the camera 110 to the queue and the angle of the camera110 with respect to the queue). Note that when the position where thequeue is to be formed is specified in advance, the position is includedin the environmental object information together with the structuralobject information, and then input.

FIGS. 3A and 3B show a relation between the position of the camera 110and the queue position. FIG. 3A is a top view of a positional relationbetween a queue line with the camera layout (A) of FIG. 2A (a lineindicating a direction in which the queue extends, corresponding to theabove-mentioned staying line, because it is also a line along which thestaying persons line up) and the camera 110. When the position of thecamera is (Xcam, Ycam), and the position of the head of the queuedetermined by the position of the cash register counter is (Xa, Ya), theposition and pose determination unit 103 can obtain a distance betweenthe camera 110 and the head of the queue from a difference between thesecoordinates. Moreover, since the position and pose determination unit103 can know the queue line from the structural object information, itcan obtain an angle between the queue line and a line connecting thecamera to the first person in the queue (the angle is hereinafterreferred to as a relative azimuth angle with respect to the line inwhich the persons line up or simply referred to as a relative azimuthangle). With the camera layout (A), the relative azimuth angle becomeszero in the view seen from above. The way in which the persons overlapwith each other in the horizontal direction of the video changesaccording to the relative azimuth angle, and thus the hiding level alsochanges. On the other hand, as shown in FIG. 4 , the camera 110 isplaced on the upper side (usually the ceiling), and a depression angle φwith respect to the first person in the queue is acquired by adifference between the height of the camera 110 and an assumed height ofthe person and a distance from the camera 110 to the head of the queue.The way in which the persons overlap with each other in the verticaldirection of the video changes according to the depression angle, andthus the hiding level also changes. Thus, the position and posedetermination unit 103 can calculate the spatial hiding level of thepersons lining up in the queue by obtaining both the depression angleand the relative azimuth angle.

Likewise, FIG. 3B shows a top view of a positional relation between thequeue line and the camera 110 with the camera layout (B) of FIG. 2A. Therelative azimuth angle is θ with the camera layout (B). Thus, theposition and pose determination unit 103 can calculate the spatialhiding level of the persons lining up in the queue by obtaining therelative azimuth angle θ and the depression angle φ.

For example, when a distance between the persons is approximated by d,and a size of the person is approximated by a cylinder having a width Wand a height H, the position and pose determination unit 103 canapproximate a ratio r (D, θ, φ) of the spatial hiding by the followingformula (1), where the distance from the camera 110 to the person is D,the depression angle is φ, and the relative azimuth angle is θ.

$\begin{matrix}{\left\lbrack {{Formula}1} \right\rbrack} &  \\{{r\left( {D,\theta,\varphi} \right)} = {{\min\left( {1,{\max\left( {{0,1} + \frac{d\cos\theta}{2D} - \frac{d\sin\theta}{W}} \right)}} \right)}{\max\left( {{0,1} - \frac{d\cos{\theta tan}\varphi}{H}} \right)}}} & (1)\end{matrix}$

The environment object dependent object dependent staying characteristicinformation storage unit 102 stores, as the environmental objectdependent object staying characteristic information, information aboutd, W, and H and information about a position and an orientation of thequeue line (the staying line) with respect to the cash register counterwhich represent the staying characteristic. The position and posedetermination unit 103 calculates the spatial hiding level by Formula(1). Alternatively, the spatial hiding level for each of D, θ, and φ maybe calculated in advance, and this spatial hiding level may be stored inthe environmental object dependent object staying characteristicinformation storage unit 102. When the environmental object informationincludes a structural object such as a cash register counter at which aqueue can be formed, the position and pose determination unit 103obtains D, θ, and φ from the environmental object information and thecamera layout, and reads the corresponding spatial hiding level from theenvironmental object dependent object staying characteristic informationstorage unit 102.

In reality, there are variations to some extent in values of thedistance d between the persons, and the sizes W and H of the persons.There are also variations in the positions where the persons stand.Thus, the position and pose determination unit 103 may treat the spatialhiding level as a distribution p (r (D, θ, φ)) instead of treating it asone value determined by D, θ, and φ. The distribution is calculated byrepeatedly calculating the spatial hiding level by varying the values ofthe distance d between the persons, the sizes W and H of the persons,and the positions where the persons stand within a certain range, andobtaining a frequency distribution. Then, representative values (mean,mode, median, etc.) of the distribution p (r (D, θ, φ)) may becalculated and stored in the environmental object dependent objectstaying characteristic information storage unit 102 to be used by theposition and pose determination unit 103.

With the camera layout example of FIG. 2A, the spatial hiding level islower with the camera layout (B) than it is with the camera layout (A).Thus, in this case, the camera layout (B) is selected by the positionand pose determination unit 103. The position and pose determinationunit 103 generates a plurality of such camera layout candidates andselects a camera layout in which the spatial hiding level becomes thelowest. Alternatively, the position and pose determination unit 103 mayobtain a plurality of camera layouts in which the spatial hiding levelbecomes less than or equal to a predetermined threshold, and select theoptimal one among the plurality of camera layouts using other indices.Other indices may be shielding by other obstacles, a resolution or anangle of the object, or a coverage ratio indicating how much the camera110 can cover a range. For example, the method disclosed in PatentLiterature 2 can be used as a method of selecting the optimal cameralayout.

The spatial hiding level has been obtained as one value so far. However,in reality, the spatial hiding level varies from place to place. Forexample, in the example of the above queue, the distance and angle fromthe camera to the head of the queue differs from those of the end of thequeue, and thus the value of the spatial hiding level also changes. Forthis reason, the position and pose determination unit 103 may calculatethe spatial hiding level for each position (X, Y) of the monitored areaand may select the camera layout based on the representative value(e.g., the mean).

In the specific example 1, the temporal hiding level has not beenconsidered. This is because in the specific example 1, the influence ofthe temporal hiding level is the same regardless of the camera layout(e.g., the temporal hiding level stays the same, because the cameralayouts (A) and (B) see the same queue). An example in which thetemporal hiding level is considered will be described later.

In the specific example 1, an example in which the structural object isthe cash register has been described. However, the structural object maybe any structural object as long as it is a structural object in which aqueue can be formed (or a queue is induced). For example, the specificexample 1 can be applied to, for example, a queue in front of an ATM(Automatic Teller Machine), a queue in front of an immigration counterat an airport, and a queue in front of a ticket machine of a station ina manner similar to the example described above.

(Specific Example 2) When a Plurality of Queues are Formed at a CashRegister which is the Structural Object of the Environmental Object

Next, a case in which the structural object of the environmental objectis a cash register counter, and a plurality of queues are formed infront of the cash register counter will be described.

FIG. 5A shows a camera layout example of the camera 110 for capturingpersons lining up at a cash register counter where a plurality of queuesare formed, and is a view seen from directly above. Also in this FIG.5A, two types of camera layouts (A′) and (B′) are shown. The cameralayout (A′) shows a camera layout for capturing persons lining up infront of one cash register counter from the front the cash registercounter, and the camera layout (B′) shows a camera layout in which twoqueues are viewed diagonally in the middle of the two queues. FIG. 5B isan example of a video of persons assumed to be captured with the cameralayout (A′), and FIG. 5C is an example of a video of persons assumed tobe captured with the camera layout (B′). In the camera layout examplesof FIG. 5A, as described in the specific example 1, the spatial hidinglevel is smaller with the camera layout (B′) than with the camera layout(A′). Further, with the camera layout (A′), another camera 110 isrequired in front of the queue in order to see the other queue. On theother hand, with the camera layout (B′), one camera 110 can cover twoqueues, and thus it can be said that the camera layout (B′) is morepreferable than the camera layout (A′) in terms of the number of cameras110.

Thus, the position and pose determination unit 103 selects the cameralayout (B′). However, when the distance between the queues is narrow, orwhen the depression angle of the camera 110 is small due to limitationssuch as the height of the ceiling, the spatial hiding level may not besufficiently lowered in some cases. In such a case, the spatial hidinglevel can be lowered by employing the camera layout described in thespecific example 1 for each queue.

(Specific Example 3) When a Plurality of Queues are Formed at a CashRegister Counter which is the Structural Object of the EnvironmentalObject, and there is a Difference in Speeds at which the Queues Move

Next, a case where the structural object of the environmental object isa cash register counter, a plurality of queues are formed in front ofthe cash register counter, and there is a difference in the speeds atwhich the queues move will be described.

FIG. 6A shows a camera layout example of the camera 110 for capturingpersons lining up at a cash register counter where a plurality of queuesare formed, and is a view seen from directly above. A difference betweenFIG. 6A and FIG. 5A is that in FIG. 6A, the left side is a normal cashregister, and the right side is an express cash register (e.g., aself-checkout where a store clerk only checks items, and a customer paysmoney with a self-payment machine). The speed at which the right queuemoves is faster than that of the left queue, and thus there is adifference in the speeds at which the queues move. Also in this FIG. 6 ,two types of camera layouts (A″) and (B″) are shown. The camera layout(A″) shows a camera layout in which the camera 110 is installed closerto the normal cash register side with respect to a center line betweenthe queues to capture both queues. Conversely, the camera layout (B″)shows a camera layout in which the camera 110 is installed closer to theexpress cash register side with respect to the center line between thequeues to capture both queues. Considering only the spatial hiding levelof the camera layout example of FIG. 6A, when a shift of the camera 110from the center line between the queues is the same for the cameralayouts (A″) and (B″), the hiding levels become the same. However, thespeed at which the queue for the normal cash register moves is slowerthan the speed at which the queue for the express cash register moves,and thus a period of time during which a hiding state continues islonger for the normal cash register than the express cash register.Thus, when the total hiding level in consideration of the temporalhiding level is calculated, the camera layout (B″) is preferable to thecamera layout (A″), because the total hiding level becomes lower withthe camera layout (B″) than with the camera layout (A″). In thefollowing descriptions, a method of calculating the temporal hidinglevel and a method of calculating the total hiding level based on thetemporal hiding level will be described.

The position and pose determination unit 103 can define the temporalhiding level by the duration of hiding (the period of time during whichthe hiding state continues). That is, the position and posedetermination unit 103 regards that the longer the duration of hiding,the greater the temporal hiding level becomes. For example, as shown inFIG. 7 , the position and pose determination unit 103 can numericallyexpress the temporal hiding level as a monotone non-decreasing functionr_(t)(t) of the duration of the hiding t which satisfies the formula(2).

[Formula 2]

0≤r _(t)(t)≤1,r _(t)(0)=0,r _(t)(t ₀)≤r _(t)(t ₁)(0≤t ₀ ≤t ₁)  (2)

In a queue, when the first person checks out and the next person movesforward, the entire queue moves forward. The hiding state is temporarilyresolved at this timing, and thus the hiding duration can be consideredas a period of time during which a stationary state of the queuecontinues. Therefore, the position and pose determination unit 103 mayobtain the distribution of the time when the queue is in a stationarystate, and use the representative value as the hiding duration. Theposition and pose determination unit 103 can calculate the temporalhiding level by r_(t)(t) using this hiding duration.

The position and pose determination unit 103 can define the total hidinglevel as a value obtained by integrating the spatial hiding level withthe temporal hiding level, and for example, the value obtained bymultiplying the hiding level by the temporal hiding level may be used asthe total hiding level. Assume that the hiding durations of the left andright queues in FIG. 6A are T₁ and T₂ (T₁>T₂), respectively, and thespatial hiding levels of the left and right queues in FIG. 6B are R₁ andR₂, respectively (where R₁>R₂).

In this case, the position and pose determination unit 103 can calculatethe total hiding level for the camera layout (A″) by averaging the twoqueues as shown in Formula (3).

$\begin{matrix}\left\lbrack {{Formula}3} \right\rbrack &  \\{\frac{1}{2}\left\{ {{R_{1}{r_{t}\left( T_{1} \right)}} + {R_{2}{r_{t}\left( T_{2} \right)}}} \right\}} & (3)\end{matrix}$

On the other hand, the position and pose determination unit 103 cancalculate the total hiding level for the camera layout (B″) as shown inFormula (4).

$\begin{matrix}\left\lbrack {{Formula}4} \right\rbrack &  \\{\frac{1}{2}\left\{ {{R_{2}{r_{t}\left( T_{1} \right)}} + {R_{1}{r_{t}\left( T_{2} \right)}}} \right\}} & (4)\end{matrix}$

Since T₁>T₂ currently holds, r_(t)(T₁)>r_(t)(T₂) holds, and the totalhiding level of Formula (3) is greater than the total hiding level ofFormula (4).

Therefore, in the camera layout example of FIG. 6A, the position andpose determination unit 103 selects the camera layout (B″). However,when the distance between the queues is narrow, or when the depressionangle of the camera is small due to limitations such as the height ofthe ceiling, the spatial hiding level may not be sufficiently lowered insome cases. In this case, the position and pose determination unit 103may make the same selection as in the specific example 2.

In this way, when there is a difference in the speed at which the queuesmove, it can be seen that it is better to position the camera 110 closerto the queue which moves faster than the other queue with respect to thecenter line between the queues. In the above descriptions, the totalhiding level is calculated using the staying duration, and then thecamera layout is selected. However, if only the speed at which the queuemoves is known for each type of the cash register as the stayingcharacteristic, it is possible to know which queue side the camera 110should be positioned closer to. Thus, the environmental object dependentobject staying characteristic information storage unit 102 may storevelocity information of the queue as the environment dependent objectstaying characteristic information, and the position and posedetermination unit 103 may determine which queue side the camera 110should be positioned closer to only from the velocity information of thequeues. Alternatively, an amount of shifting the camera 110 from thecenter line between the queues may be calculated in advance based on thespeed difference between the queues, and a result of the calculation maybe included in the environment dependent object staying characteristicinformation to be stored in the environmental object dependent objectstaying characteristic information storage unit 102. Then, when it isfound that there is a speed difference between adjacent queues from theenvironmental object information, the position and pose determinationunit 103 may read information about the amount of shifting the camera110 according to the speed difference from the environmental objectdependent object staying characteristic information storage unit 102 anddetermine the video acquisition unit layout information.

(Specific Example 4) When the Structural Objects of the EnvironmentalObject are a Wall and a Fixture (Shelf) and the Placed Objects are Items

FIG. 8A shows a camera layout example of the camera 110 for capturingpersons in the vicinity of a commodity shelf, and is a view seen fromdirectly above. Here, in addition to a shelf as the structural object,items (in this case, books) which are placed objects placed on thestructural object are also presented. On the other hand, the other sideof the shelf across the passage is a wall. Also in this FIG. 8A, twotypes of camera layouts (C) and (D) are shown. The camera layout (C)shows a camera layout for capturing a video from a position closer tothe shelf where the books are placed, and the camera layout (D) is acamera layout for capturing a video from a position closer to the wall.FIG. 8B is an example of a video of persons assumed to be captured withthe camera layout (C), and FIG. 8C is an example of a video of personsassumed to be captured with the camera layout (D). FIGS. 8B and 8C showhow the situation where two persons staying side by side in front of theshelf where the books are placed is captured. Considering that shoppersare likely to stay in front of book shelves, the situation of staying inthis way is likely to occur. Therefore, in the example of the cameralayout of FIG. 8A, it can be said that the camera layout (D) ispreferable to the camera layout (C), because the spatial hiding level ofthe camera layout (D) can be made smaller than that of the camera layout(C). In the following descriptions, a method of calculating the spatialhiding level in each case of the camera layout (C) and (D) will bedescribed.

First, the position and pose determination unit 103 generates two cameralayouts (C) and (D) as candidates for the camera layout of the camera110 (the position and pose of the camera 110). The position and posedetermination unit 103 determines, from the camera layouts (C) and (D)and the environmental object information, a positional relation betweenthe structural object in the monitored area and placed objects (in thiscase, the shelf and the items) and the camera 110 in the monitored areaas shown in FIG. 8A. Then, the position and pose determination unit 103can determine a position where persons are likely to stay, and like inthe specific example 1, determine the positional relation between thisposition and the camera 110 (a distance from the camera 110 to thestaying person or an angle of the camera 110 with respect to a directionin which the staying person lines up). Thus, like in the specificexample 1, the position and pose determination unit 103 can obtain therelative azimuth angle and the depression angle with respect to thedirection in which the staying person lines up.

Next, the position and pose determination unit 103 obtains the spatialhiding level for the depression angle and the relative azimuth angleobtained as above from the environmental object dependent object stayingcharacteristic information. At this time, like in the specific example1, the environmental object dependent object staying characteristicinformation includes a parameter such as a distance between personswhich is necessary for obtaining the spatial hiding level. The positionand pose determination unit 103 calculates the spatial hiding levelusing this parameter. This spatial hiding level can be calculated in thesame manner as in the specific example 1, although the value of theparameter such as the distance between persons differ from that for thequeue in the specific example 1. Alternatively, as in the specificexample 1, the information about the hiding level calculated for each ofthe distance to the staying position, the depression angle of thecamera, and the relative azimuth angle may be stored in advance in theenvironmental object dependent object staying characteristic informationstorage unit 102 as the environmental object dependent object stayingcharacteristic information. The characteristic of this hiding levelvaries depending on the type of items placed on the shelf. For thisreason, the hiding level is obtained for each placed object placed onthe shelf and stored, and the position and pose determination unit 103uses the corresponding hiding level information according to theinformation of the placed object included in the input environmentalobject information. The staying duration varies depending on the placedobject on the shelf. A case where this varying staying duration isconsidered will be described later.

In the camera layout example shown in FIG. 8 , the position and posedetermination unit 103 selects the camera layout (D), because the valueof the spatial hiding level of the camera layout (D) is smaller thanthat of the camera layout (C). Like in the specific example 1, in thespecific example 4, the position and pose determination unit 103generates a plurality of camera layouts, and determines the cameralayout based on the calculated value of the spatial hiding level foreach of the camera layouts.

(Specific Example 5) When the Structural Object of the EnvironmentalObject is a Fixture (Shelf) and the Placed Objects are Items

FIG. 9A shows a camera layout example of the camera 110 for capturingpersons in the vicinity of a commodity shelf, and is a view seen fromdirectly above. Here, in addition to a shelf as a structural object,items (in this case, book and everyday use items) which are placedobjects placed on the structural object are also presented. Also in thisFIG. 9A, two types of camera layouts (C′) and (D′) are shown. The cameralayout (C′) shows a camera layout for capturing a video from a positioncloser to the shelf where the books are placed, and the camera layout(D′) is a camera layout for capturing a video from a position closer toa shelf where everyday use items are arrange. FIG. 9B is an example of avideo of persons assumed to be captured with the camera layout (C′), andFIG. 9C is an example of a video of persons assumed to be captured withthe camera layout (D′). FIGS. 9B and 9C show how the situation where twopersons staying side by side in front of the shelf where the books areplaced and the shelf where the everyday use items are placed iscaptured. The spatial hiding level is the same between the cameralayouts (C′) and (D′) in the videos shown in FIGS. 9B and 9C. However, aperson tends to stay for a longer time in front of a book shelf than infront of an everyday use item shelf, and thus a period of time duringwhich the hiding state continues is longer for the book shelf than theeveryday item shelf. Thus, considering the temporal hiding level, it canbe said that the camera layout (D′) is preferable. In the followingdescriptions, a method of calculating the temporal hiding level and amethod of calculating the total hiding level using the temporal hidinglevel will be described in each case of the camera layout (C′) and (D′).

Basically, the calculation of the temporal hiding level is the same asin the case of the specific example 3. That is, the position and posedetermination unit 103 can obtain the temporal hiding level as afunction of the hiding duration. However, the method of obtaining thehiding duration differs from that for the queue. In the case of staying,the hiding duration varies depending on an occurrence frequency ofstaying and a staying time per stay. For example, assuming that thestaying state occurs uniformly, that the number of occurrences ofstaying per unit time is N, and that the staying duration per staying isT, an average hiding duration can be approximated by Formula (5).

$\begin{matrix}\left\lbrack {{Formula}5} \right\rbrack &  \\{\max\left( {0,{T - \frac{1}{N}}} \right)} & (5)\end{matrix}$

Here, 1/N can be regarded as a time interval in which the stayingoccurs. In reality, the occurrence of the staying may not follow auniform distribution. Moreover, the staying time is not a constant valueand instead it usually has some distribution. Thus, the position andpose determination unit 103 may calculate the hiding duration inconsideration of such factors. Specifically, the position and posedetermination unit 103 may actually measure distributions of occurrencetime intervals between the staying and staying times, and calculatehiding duration by simulation based on the measured distribution. Thatis, the position and pose determination unit 103 may generate a stayingsituation that follows the actually measured distribution and obtain thetime during which the hiding state continues. The hiding durationacquired in this manner changes depending on the placed object. Thus,the hiding duration may be obtained for each placed object, and thehiding duration for each placed object may be stored in theenvironmental object dependent object staying characteristic informationstorage unit 102 as the environment dependent object stayingcharacteristic information.

Then, like in the specific example 3, the position and posedetermination unit 103 can calculate total hiding level information bymultiplying the temporal hiding level by the spatial hiding level thusobtained. The position and pose determination unit 103 determines thecamera layout based on the total hiding level information obtained asabove. In the camera layout example of FIG. 9A, the camera layout (D′)is selected for the same reason as in the specific example 3. In thisway, in the case of a passage that is sandwiched between shelves havingstaying levels different from each other such as the shelves havingstaying durations different from each other, it is preferable to installthe camera 110 closer to the shelf where there is less staying withrespect to the center, because the hiding level as a whole can be madelower. That is, the position and pose determination unit 103 may be atleast installed in a camera layout in which the camera 110 is placedcloser to the shelf where persons are less likely to stay than the othershelf with respect to the center of the passage, and a camera layout inwhich the hiding level is small among these camera layouts may beselected.

In the above descriptions, the total hiding level is calculated usingthe staying duration and the staying frequency to select the cameralayout. However, if only the hiding duration determined by the stayingduration and the staying frequency is known for each placed objectplaced on the shelf as the staying characteristic, it is possible toknow which shelf should the camera 110 be placed closer to. Thus, thehiding duration information about the placed object on the shelf may bestored in the environmental object dependent object stayingcharacteristic information storage unit 102 as the environment dependentobject staying characteristic information, and the position and posedetermination unit 103 may directly determine which shelf side thecamera should be positioned closer to only from this information.Alternatively, the temporal hiding level itself acquired by the hidingduration may be included in the environment dependent object stayingcharacteristic information as an index (staying degree) indicatinglikeliness of an occurrence of the staying. The amount of shifting thecamera 110 from the center line of the passage may be calculated inadvance, and a result of the calculation may be included in theenvironment dependent object staying characteristic information. When itis found from the environmental object information that there is adifference between the staying level of one side of the passage and thatof the other side of the passage, the position and pose determinationunit 103 may read information about the amount of shifting the camera110 from the environmental object dependent object stayingcharacteristic information storage unit 102 according to the differencein the staying levels to determine the camera layout information.

As described above, in the first embodiment, the video monitoringapparatus 100 determines the position and the pose of the videoacquisition unit 110 in such a way that the hiding level caused byoverlapping of objects becomes low based on environmental objectinformation indicating an environmental object including a structuralobject present in the monitored area and a placed object placed in thestructural object and staying characteristic information indicating astaying characteristic of the object determined depending on theenvironmental object.

As described above, according to the first embodiment, the videomonitoring apparatus 100 considers not only the information about thestatic environmental object in the monitoring area but also the stayingcharacteristic of the object determined depending on the environmentalobject to determine the position and pose of the video acquisition unit.For this reason, even under the situation where objects overlap witheach other due to the staying of the objects determined depending on theenvironmental object, and the objects are hidden, it is possible toperform appropriate video monitoring. As more appropriate videomonitoring can be made possible, it is possible to more appropriatelymonitor the monitored object and perform object detection/trackingprocessing, thereby providing a more appropriate analysis result.

Second Embodiment

Referring to FIG. 10 , a configuration example of a video monitoringapparatus 200 according to a second embodiment of the present disclosureis shown. The video monitoring apparatus 200 includes a videoacquisition unit 210, an analysis and display unit 111, an environmentalobject information storage unit 101, an environmental object dependentobject staying characteristic information storage unit 102, a positionand pose determination unit 203, and a position and pose control unit204.

The position and pose determination unit 203 generates video acquisitionunit layout information based on environmental object information of amonitoring area stored in the environmental object information storageunit 101 and environment object dependent object staying characteristicinformation stored in the environmental object dependent object stayingcharacteristic information storage unit 102, and outputs the generatedvideo acquisition unit layout information to the position and posecontrol unit 204.

The position and pose control unit 204 generates position and posecontrol information based on the video acquisition unit layoutinformation output from the position and pose determination unit 203,and outputs it to the video acquisition unit 210.

The video acquisition unit 210 can change the position and pose byexternal control. The video acquisition unit 210 changes the positionand pose according to the position and pose control information outputfrom the position and pose control unit 204, and outputs the capturedvideo to the analysis and display unit 111.

Next, an operation of the video monitoring apparatus 200 shown in FIG.10 will be described.

Operations of the environmental object information storage unit 101 andthe environmental object dependent object staying characteristicinformation storage unit 102 are the same as those of the videomonitoring apparatus 100 of FIG. 1 . However, the environmental objectinformation may be set for each time. For example, if the number ofcounters in front of the cash register or the number of counters at anairport which open is determined in advance according to time of day,the information about the counter which opens for each time of day maybe included in the structural object information, and the position andpose determination unit 203 may obtain the layout of the videoacquisition unit 210 for time of day using the information about thecounter and outputs the video acquisition unit layout information. Thevideo acquisition unit layout information generated by the position andpose determination unit 203 is input to the position and pose controlunit 204.

The position and pose control unit 204 generates the position and posecontrol information for controlling the video acquisition unit 210 basedon the video acquisition unit layout information input from the positionand pose determination unit 203. The position and pose controlinformation generated by the position and pose control unit 204 isoutput to the video acquisition unit 210.

The video acquisition unit 210 can change the position and pose byexternal control. For example, the video acquisition unit 210 may be adevice that can float in the air and capture a video like a UAV(Unmanned Aerial Vehicle). Alternatively, the video acquisition unit 210may be a camera placed on a wall or a ceiling, and may be a camera whichcan move on an installed rail. Further alternatively, the videoacquisition unit 210 may be a camera mounted on a robot which can move.The video acquisition unit 210 changes the position and pose accordingto the position and pose control information input from the position andpose control unit 204, and captures a video. Then, the video acquisitionunit 210 outputs the captured video to the analysis and display unit111.

The operation of the analysis and display unit 111 is the same as thatof the video monitoring apparatus 100 of FIG. 1 .

As described above, according to the second embodiment, the videomonitoring apparatus 200 sets the environmental object information foreach time of day. Therefore, the video monitoring apparatus 200 canappropriately change the position and pose of the video acquisition unit210 according to the situation of the structural object which changesaccording to the time of day, thereby making it possible to provide amore appropriate monitoring videos and analysis results.

Third Embodiment

Referring to FIG. 11 , a configuration example of a video monitoringapparatus 300 according to a third embodiment of the present disclosureis shown. The video monitoring apparatus 300 includes a videoacquisition unit 210, an analysis and display unit 111, an environmentalobject information storage unit 101, an environmental object dependentobject staying characteristic information storage unit 102, a positionand pose determination unit 303, a position and pose control unit 204,and an object state acquisition unit 305.

The object state acquisition unit 305 acquires a state of an object tobe monitored and outputs object state information indicating the stateof the object to the position and pose determination unit 303.

The position and pose determination unit 303 generates the above videoacquisition unit layout information based on the environmental objectinformation of the monitoring area stored in the environmental objectinformation storage unit 101, environmental object dependent objectstaying characteristic information stored in the environmental objectdependent object staying characteristic information storage unit 102,and the object state information output from the object stateacquisition unit 305, and outputs the generated video acquisition unitlayout information to the position and pose control unit 204.

The other configurations of the video monitoring apparatus 200 are thesame as those of the video monitoring apparatus 200 shown in FIG. 10 .

Next, an operation of the video monitoring apparatus 300 shown in FIG.11 will be described.

The object state acquisition unit 305 is means for observing a state ofan object in the monitored area, and observes the state of the objectsuch as the number and movements of the object, and a structure (e.g., aqueue) formed by the object. Specifically, the object state acquisitionunit 305 may be a sensor installed at a fixed position of the monitoredarea or an imaging device which acquires a video. The sensor may be asensor (a human detection sensor if the object is a person) whichdetects an object, or a sensor which catches signals such as RFID (RadioFrequency IDentification), WiFi (Wireless Fidelity), BlueTooth(registered trademark) to estimate whether an object is present and thenumber of objects near the sensor. Alternatively, the sensor may be adevice which collects acoustic information such as a microphone andanalyzes the collected acoustic information to know the state of theobject. When the object state acquisition unit 305 is a device whichacquires a video, the device may analyze the object in the video andknow the number of objects and the state of the object. Alternatively,the video acquisition unit 210 itself may play a role of the device.Various known methods may be used as a method of analyzing the objectand knowing the state of the object. The object state informationacquired by the object state acquisition unit 305 is input to theposition and pose determination unit 303.

Among the operations of the position and pose determination unit 303, anoperation when the object state information is not input from the objectstate acquisition unit 305 is the same as the operation of the positionand pose determination unit 203 of FIG. 10 . On the other hand, when theobject state information is input from the object state acquisition unit305, the position and pose determination unit 303 corrects the videoacquisition unit layout information based on the object stateinformation. For example, when the number of queues varies according tocircumstances in an area where a plurality of queues can be formed, theposition and pose determination unit 303 knows in advance positions atwhich the queues are formed based on the object state information tocalculate a hiding level and generate the video acquisition unit layoutinformation. Alternatively, the position and pose determination unit 303may weight, according to reliability of the object state information,the hiding level obtained without the object state information and thehiding level obtained based on the object state information, obtain thehiding level by obtaining a weighted average, and then generates thevideo acquisition unit layout information. The video acquisition unitlayout information generated by the position and pose determination unit303 is output to the position and pose control unit 204.

The operations of the position and pose control unit 204, the videoacquisition unit 210, and the analysis and display unit 111 are the sameas those of the video monitoring apparatus 200 shown in FIG. 10 .

As described above, according to the third embodiment, the videomonitoring apparatus 300 acquires the state of the object and changesthe position and pose of the video acquisition unit 210 based on thestate of the object. By doing so, the video monitoring apparatus 300 canappropriately change the position and pose of the video acquisition unit110 according to the state of the object which changes depending on thesituation, thereby making it possible to provide more appropriate videomonitoring and analysis result.

Further, each element shown in the drawings as functional blocks thatperform various processing can be formed of a CPU (Central ProcessingUnit), a memory, and other circuits in hardware and may be implementedby programs loaded into the memory in software. Those skilled in the artwill therefore understand that these functional blocks may beimplemented in various ways by only hardware, only software, or thecombination thereof without any limitation.

The above program can be stored and provided to a computer using anytype of non-transitory computer readable media. Non-transitory computerreadable media include any type of tangible storage media. Examples ofnon-transitory computer readable media include magnetic storage media(such as floppy disks, magnetic tapes, hard disk drives, etc.), opticalmagnetic storage media (e.g. magneto-optical disks), CD-ROM (CompactDisc-Read Only Memory), CD-R (CD-Recordable), CD-R/W (CD-ReWritable),and semiconductor memories (such as mask ROM, PROM (Programmable ROM),EPROM (Erasable PROM), flash ROM, RAM (Random Access Memory), etc.). Theprogram may be provided to a computer using any type of transitorycomputer readable media. Examples of transitory computer readable mediainclude electric signals, optical signals, and electromagnetic waves.Transitory computer readable media can provide the program to a computervia a wired communication line (e.g. electric wires, and optical fibers)or a wireless communication line.

Although the present disclosure has been described with reference to theembodiments, the present disclosure is not limited to these embodiments.Various changes that can be understood by those skilled in the artwithin the scope of the present disclosure can be made to theconfigurations and the details of the present disclosure.

The whole or part of the embodiments disclosed above can be describedas, but not limited to, the following supplementary notes.

(Supplementary Note 1)

A video monitoring apparatus for monitoring inside a monitored areausing a video acquisition unit, the video monitoring apparatuscomprising:

-   -   a position and pose determination unit configured to determine a        position and a pose of the video acquisition unit in such a way        that a level of hiding caused by overlapping of objects becomes        low based on environmental object information indicating an        environmental object including a structural object present in        the monitored area and a placed object placed in or on the        structural object, and staying characteristic information        indicating a staying characteristic of the object determined        depending on the environmental object; and    -   an analysis and display unit configured to perform at least one        of analyzing a video captured by the video acquisition unit at        the position and the pose determined by the position and pose        determination unit and displaying the video.

(Supplementary Note 2)

The video monitoring apparatus according to Supplementary note 1,wherein

-   -   the video acquisition unit can change the position and the pose        by external control,    -   the video monitoring apparatus further comprises a position and        pose control unit configured to generate control information for        controlling the video acquisition unit based on the position and        the pose determined by the position and pose determination unit,        and    -   the video acquisition unit changes the position and the pose        based on the control information generated by the position and        pose control unit.

(Supplementary Note 3)

The video monitoring apparatus according to Supplementary note 1,further comprising an object state acquisition unit configured toacquire a state of the object, wherein

-   -   the position and pose determination unit determines the position        and the pose of the video acquisition unit also using the state        of the object acquired by the object state acquisition unit.

(Supplementary Note 4)

The video monitoring apparatus according to any one of Supplementarynotes 1 to 3, wherein

-   -   the staying characteristic information includes at least one of        information about a position and an orientation of a staying        line that is a line along which a staying object lines up and is        determined depending on the environmental object, and        information about a staying duration of the staying object, and    -   the position and pose determination unit calculates the level of        hiding by calculating a spatial hiding level based on a relative        azimuth angle and a distance between the staying line and the        video acquisition unit, and a depression angle of the video        acquisition unit, or by calculating a temporal hiding level        based on the staying duration.

(Supplementary Note 5)

The video monitoring apparatus according to any one of Supplementarynotes 1 to 3, wherein when the environmental object information includesinformation indicating a structural object which induces generation of aplurality of queues, and the staying characteristic informationindicates the staying line along which the staying object lines upcorresponding to each of the plurality of queues formed in front of thestructural object, the position and pose determination unit determinesto install the video acquisition unit between the staying lines.

(Supplementary Note 6)

The video monitoring apparatus according to any one of Supplementarynotes 1 to 3, wherein when the environmental object information includesinformation indicating a structural object which induces generation of aplurality of queues, the staying characteristic information indicatesthe staying line along which the staying object lines up correspondingto each of the plurality of queues formed in front of the structuralobject, and speeds at which the queues move differ from each other, theposition and pose determination unit determines to install the videoacquisition unit closer to a side of the queue which moves faster thananother queue with respect to a center between the queues.

(Supplementary Note 7)

The video monitoring apparatus according to any one of Supplementarynotes 1 to 3, wherein when the environmental object information includesinformation indicating a passage and information indicating placedobjects placed on the structural objects on both sides of the passage,and the staying characteristic information indicates that a stayinglevel of the object differs from that of another object, the positionand pose determination unit determines to install the video acquisitionunit to a position closer to the side in which the staying level issmaller than that of the other side with respect to a center of thepassage.

(Supplementary Note 8)

A method of controlling video monitoring apparatus for monitoring insidea monitored area using a video acquisition unit, the method comprising:

-   -   determining a position and a pose of the video acquisition unit        in such a way that a level of hiding caused by overlapping of        objects becomes low based on environmental object information        indicating an environmental object including a structural object        present in the monitored area and a placed object placed in or        on the structural object, and staying characteristic information        indicating a staying characteristic of the object determined        depending on the environmental object; and    -   performing at least one of analyzing a video captured by the        video acquisition unit at the determined position and the        determined pose and displaying the video.

(Supplementary Note 9)

The method according to Supplementary note 8, wherein

-   -   the video acquisition unit can change the position and the pose        by external control,    -   the method further comprises generating control information for        controlling the video acquisition unit based on the position and        the pose determined in the determination of the position and the        pose, and    -   the video acquisition unit changes the position and the pose        based on the control information generated by the position and        pose control unit.

(Supplementary Note 10)

The video monitoring apparatus according to Supplementary note 8,further comprising acquiring a state of the object, wherein

-   -   in the determination of the position and the pose, the position        and the pose of the video acquisition unit are determined also        using the state of the object acquired by the object state        acquisition unit.

(Supplementary Note 11)

The video monitoring apparatus according to any one of Supplementarynotes 8 to 10, wherein

-   -   the staying characteristic information includes at least one of        information about a position and an orientation of a staying        line that is a line along which a staying object lines up and is        determined depending on the environmental object and information        about a staying duration of the staying object, and    -   in the determination of the position and pose, the level of        hiding is calculated by calculating a spatial hiding level based        on a relative azimuth angle and a distance between the staying        line and the video acquisition unit, and a depression angle of        the video acquisition unit, or by calculating a temporal hiding        level based on the staying duration.

(Supplementary Note 12)

The method according to any one of Supplementary notes 8 to 10, whereinwhen the environmental object information includes informationindicating a structural object which induces generation of a pluralityof queues, and the staying characteristic information indicates thestaying line along which the staying object lines up corresponding toeach of the plurality of queues formed in front of the structuralobject, in the determination of the position and the pose, it isdetermined to install the video acquisition unit between the stayinglines.

(Supplementary Note 13)

The method according to any one of Supplementary notes 8 to 10, whereinwhen the environmental object information includes informationindicating a structural object which induces generation of a pluralityof queues, the staying characteristic information indicates the stayingline along which the staying object lines up corresponding to each ofthe plurality of queues formed in front of the structural object, andspeeds at which the queues move differ from each other, in thedetermination of the position and the pose, it is determined to installthe video acquisition unit closer to a side of the queue which movesfaster than another queue with respect to a center between the queues.

(Supplementary Note 14)

The method according to any one of Supplementary notes 8 to 10, whereinwhen the environmental object information includes informationindicating a passage and information indicating placed objects placed onthe structural objects on both sides of the passage, and the stayingcharacteristic information indicates that a staying level of the objectdiffers from that of another object, in the determination of theposition and the pose, it is determined to install the video acquisitionunit to a position closer to the side in which the staying level issmaller than that of the other side with respect to a center of thepassage.

(Supplementary Note 15)

A non-transitory computer readable medium storing a program for causinga computer to monitor inside a monitored area using a video acquisitionunit, the program the computer to execute:

-   -   a procedure of determining a position and an pose of the video        acquisition unit in such a way that a level of hiding caused by        overlapping of objects becomes low based on environmental object        information indicating an environmental object including a        structural object present in the monitored area and an placed        object placed in the structural object and staying        characteristic information indicating a staying characteristic        of the object to be monitored determined depending on the        environmental object; and    -   a procedure of performing at least one of analyzing a video        captured by the video acquisition unit at the determined        position and the determined pose and displaying the video.

INDUSTRIAL APPLICABILITY

The video monitoring apparatus according to the present disclosure canbe used as a video monitoring system which monitors various placesindoors and outdoors, analyzes an object therein, issues an alert when asuspicious state or a dangerous state is detected. In particular, thevideo monitoring apparatus according to the present disclosure can beused for monitoring an area in which a plurality of objects to bemonitored are present and where staying occurs in a specific place.

REFERENCE SIGNS LIST

-   -   101 ENVIRONMENTAL OBJECT INFORMATION STORAGE UNIT    -   102 ENVIRONMENTAL OBJECT DEPENDENT OBJECT STAYING CHARACTERISTIC        INFORMATION STORAGE UNIT    -   103 POSITION AND POSE DETERMINATION UNIT    -   110 VIDEO ACQUISITION UNIT    -   111 ANALYSIS AND DISPLAY UNIT    -   203 POSITION AND POSE DETERMINATION UNIT    -   204 POSITION AND POSE CONTROL UNIT    -   210 VIDEO ACQUISITION UNIT    -   303 POSITION AND POSE DETERMINATION UNIT    -   305 OBJECT STATE ACQUISITION UNIT

1. A video monitoring apparatus for monitoring inside a monitored areausing at least one video acquisition unit, the video monitoringapparatus comprising: at least one memory storing instructions; and atleast one processor configured to execute the instructions to: determinea position and a pose of the at least one video acquisition unit toreduce hiding, caused by overlapping among different objectssimultaneously appearing in a view of the at least one video acquisitionunit, to become lower than a predetermined threshold level based onenvironmental object information and staying characteristic information;generate control information for controlling the at least one videoacquisition unit based on the position and the pose determined; changethe position and the pose of the at least video acquisition unit basedon the control information; and perform at least one of analyzing avideo captured by the at least one video acquisition unit at thedetermined position and the determined pose, displaying the video andissuing an alert in accordance with a result of analyzing the video,wherein the environmental object information indicates an environmentalobject including a structural object present in the monitored area and aplaced object placed in or on the structural object, and the stayingcharacteristic information indicates a staying characteristic of eachdifferent object determined depending on the environmental object. 2.The video monitoring apparatus according to claim 1, wherein the atleast one processor is configured to execute the instructions to furtheracquire a state of each different object, and the position and the poseof the at least one video acquisition unit are determined by furtherusing the state of each different object.
 3. The video monitoringapparatus according to claim 1, wherein the staying characteristicinformation includes at least one of information about a position and anorientation of a staying line that is a line along which a stayingobject lines up and is determined depending on the environmental object,and information about a staying duration of the staying object, and theat least one processor calculates a level of the hiding by calculating aspatial hiding level based on a relative azimuth angle and a distancebetween the staying line and the at least one video acquisition unitand, a depression angle of the at least one video acquisition unit, orby calculating a temporal hiding level based on the staying duration. 4.The video monitoring apparatus according to claim 3, wherein when theenvironmental object information includes information indicating thestructural object which induces generation of a plurality of queues, andthe staying characteristic information indicates the staying line alongwhich the staying object lines up corresponding to each of the pluralityof queues formed in front of the structural object, the at least oneprocessor determines to position the at least one video acquisition unitbetween the staying line corresponding to each of the plurality ofqueues.
 5. The video monitoring apparatus according to claim 3, whereinwhen the environmental object information includes informationindicating the structural object which induces generation of a pluralityof queues, the staying characteristic information indicates the stayingline along which the staying object lines up corresponding to each ofthe plurality of queues formed in front of the structural object, andspeeds at which the queues move differ from each other, the at least oneprocessor determines to position the at least one video acquisition unitcloser to a side of the queue which moves faster than another queue withrespect to a center between the queues.
 6. The video monitoringapparatus according to claim 1, wherein when the environmental objectinformation includes information indicating a passage and informationindicating additional placed objects placed on the structural objects onboth sides of the passage, and the staying characteristic informationindicates that a staying level of each different object differs from oneanother, the at least one processor determines to position the at leastone video acquisition unit at a position closer to a side in which thestaying level is smaller than that of another side with respect to acenter of the passage.
 7. The video monitoring apparatus according toclaim 1, wherein the hiding is caused by overlapping among the differentobjects simultaneously appearing in a frame of the at least one videoacquisition unit.
 8. A video monitoring method for monitoring inside amonitored area using at least one video acquisition unit, the videomonitoring method comprising: determining, by a processor, a positionand a pose of the at least one video acquisition unit to reduce hiding,caused by overlapping among different objects simultaneously appearingin a view of the at least one video acquisition unit, to become lowerthan a predetermined threshold level based on environmental objectinformation and staying characteristic information; generating, by theprocessor, control information for controlling the at least one videoacquisition unit based on the position and the pose determined; changethe position and the pose of the at least video acquisition unit basedon the control information; and performing, by the processor, at leastone of analyzing a video captured by the at least one video acquisitionunit at the determined position and the determined pose, displaying thevideo and issuing an alert in accordance with a result of analyzing thevideo, wherein the environmental object information indicates anenvironmental object including a structural object present in themonitored area and a placed object placed in or on the structuralobject, and the staying characteristic information indicates a stayingcharacteristic of each different object determined depending on theenvironmental object.
 9. The video monitoring method according to claim8, further comprising acquiring, by the processor, a state of eachdifferent object, wherein the position and the pose of the at least onevideo acquisition unit are determined by further using the state of eachdifferent object.
 10. The video monitoring method according to claim 8,wherein the staying characteristic information includes at least one ofinformation about a position and an orientation of a staying line thatis a line along which a staying object lines up and is determineddepending on the environmental object, and information about a stayingduration of the staying object, and the video monitoring method furthercomprises calculating, by the processor, a level of the hiding bycalculating a spatial hiding level based on a relative azimuth angle anda distance between the staying line and the at least one videoacquisition unit and, a depression angle of the at least one videoacquisition unit, or by calculating a temporal hiding level based on thestaying duration.
 11. The video monitoring method according to claim 10,further comprising, when the environmental object information includesinformation indicating the structural object which induces generation ofa plurality of queues, and the staying characteristic informationindicates the staying line along which the staying object lines upcorresponding to each of the plurality of queues formed in front of thestructural object, determining, by the processor, to position the atleast one video acquisition unit between the staying line correspondingto each of the plurality of queues.
 12. The video monitoring methodaccording to claim 10, further comprising, when the environmental objectinformation includes information indicating the structural object whichinduces generation of a plurality of queues, the staying characteristicinformation indicates the staying line along which the staying objectlines up corresponding to each of the plurality of queues formed infront of the structural object, and speeds at which the queues movediffer from each other, the at least one processor determining, by theprocessor, to position the at least one video acquisition unit closer toa side of the queue which moves faster than another queue with respectto a center between the queues.
 13. The video monitoring methodaccording to claim 8, further comprising, when the environmental objectinformation includes information indicating a passage and informationindicating additional placed objects placed on the structural objects onboth sides of the passage, and the staying characteristic informationindicates that a staying level of each different object differs from oneanother, determining, by the processor, to position the at least onevideo acquisition unit at a position closer to a side in which thestaying level is smaller than that of another side with respect to acenter of the passage.
 14. The video monitoring method according toclaim 8, wherein the hiding is caused by overlapping among the differentobjects simultaneously appearing in a frame of the at least one videoacquisition unit.
 15. A non-transitory computer-readable data storagemedium storing program code executable by a processor to performprocessing for monitoring inside a monitored area using at least onevideo acquisition unit, the processing comprising: determining aposition and a pose of the at least one video acquisition unit to reducehiding, caused by overlapping among different objects simultaneouslyappearing in a view of the at least one video acquisition unit, tobecome lower than a predetermined threshold level based on environmentalobject information and staying characteristic information; generatingcontrol information for controlling the at least one video acquisitionunit based on the position and the pose determined; change the positionand the pose of the at least video acquisition unit based on the controlinformation; and performing at least one of analyzing a video capturedby the at least one video acquisition unit at the determined positionand the determined pose, displaying the video and issuing an alert inaccordance with a result of analyzing the video, wherein theenvironmental object information indicates an environmental objectincluding a structural object present in the monitored area and a placedobject placed in or on the structural object, and the stayingcharacteristic information indicates a staying characteristic of eachdifferent object determined depending on the environmental object. 16.The non-transitory computer-readable data storage medium according toclaim 15, wherein the processing further comprises acquiring a state ofeach different object, wherein the position and the pose of the at leastone video acquisition unit are determined by further using the state ofeach different object.
 17. The non-transitory computer-readable datastorage medium according to claim 15, wherein the staying characteristicinformation includes at least one of information about a position and anorientation of a staying line that is a line along which a stayingobject lines up and is determined depending on the environmental object,and information about a staying duration of the staying object, and theprocessing further comprises calculating a level of the hiding bycalculating a spatial hiding level based on a relative azimuth angle anda distance between the staying line and the at least one videoacquisition unit and, a depression angle of the at least one videoacquisition unit, or by calculating a temporal hiding level based on thestaying duration.
 18. The non-transitory computer-readable data storagemedium according to claim 17, wherein the processing further comprises,when the environmental object information includes informationindicating the structural object which induces generation of a pluralityof queues, and the staying characteristic information indicates thestaying line along which the staying object lines up corresponding toeach of the plurality of queues formed in front of the structuralobject, determining to position the at least one video acquisition unitbetween the staying line corresponding to each of the plurality ofqueues.
 19. The non-transitory computer-readable data storage mediumaccording to claim 17, wherein the processing further comprises, whenthe environmental object information includes information indicating thestructural object which induces generation of a plurality of queues, thestaying characteristic information indicates the staying line alongwhich the staying object lines up corresponding to each of the pluralityof queues formed in front of the structural object, and speeds at whichthe queues move differ from each other, the at least one processordetermining to position the at least one video acquisition unit closerto a side of the queue which moves faster than another queue withrespect to a center between the queues.
 20. The non-transitorycomputer-readable data storage medium according to claim 15, wherein theprocessing further comprises, when the environmental object informationincludes information indicating a passage and information indicatingadditional placed objects placed on the structural objects on both sidesof the passage, and the staying characteristic information indicatesthat a staying level of each different object differs from one another,determining to position the at least one video acquisition unit at aposition closer to a side in which the staying level is smaller thanthat of another side with respect to a center of the passage.